Development and validation of a survival prediction model in elder patients with community-acquired pneumonia: a MIMIC-population-based study

Background To develop a prediction model predicting in-hospital mortality of elder patients with community-acquired pneumonia (CAP) admitted to the intensive care unit (ICU). Methods In this cohort study, data of 619 patients with CAP aged ≥ 65 years were obtained from the Medical Information Mart for Intensive Care III (MIMIC III) 2001–2012 database. To establish the robustness of predictor variables, the sample dataset was randomly partitioned into a training set group and a testing set group (ratio: 6.5:3.5). The predictive factors were evaluated using multivariable logistic regression, and then a prediction model was constructed. The prediction model was compared with the widely used assessments: Sequential Organ Failure Assessment (SOFA), Pneumonia Severity Index (PSI), systolic blood pressure, oxygenation, age and respiratory rate (SOAR), CURB-65 scores using positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), area under the curve (AUC) and 95% confidence interval (CI). The decision curve analysis (DCA) was used to assess the net benefit of the prediction model. Subgroup analysis based on the pathogen was developed. Results Among 402 patients in the training set, 90 (24.63%) elderly CAP patients suffered from 30-day in-hospital mortality, with the median follow-up being 8 days. Hemoglobin/platelets ratio, age, respiratory rate, international normalized ratio, ventilation use, vasopressor use, red cell distribution width/blood urea nitrogen ratio, and Glasgow coma scales were identified as the predictive factors that affect the 30-day in-hospital mortality. The AUC values of the prediction model, the SOFA, SOAR, PSI and CURB-65 scores, were 0.751 (95% CI 0.749–0.752), 0.672 (95% CI 0.670–0.674), 0.607 (95% CI 0.605–0.609), 0.538 (95% CI 0.536–0.540), and 0.645 (95% CI 0.643–0.646), respectively. DCA result demonstrated that the prediction model could provide greater clinical net benefits to CAP patients admitted to the ICU. Concerning the pathogen, the prediction model also reported better predictive performance. Conclusion Our prediction model could predict the 30-day hospital mortality in elder patients with CAP and guide clinicians to identify the high-risk population.


Introduction
Community-acquired pneumonia (CAP), defined as pneumonia acquired outside the hospital, is one of the most common infectious diseases in clinical practice [1,2]. The incidence of pneumonia increases with age, with a 10 times higher hospitalization rate in patients aged 65 years and older (about 2000 per 100,000 per year) than in the younger population [3,4]. CAP remains a common cause of intensive care unit (ICU) admissions and in-hospital mortality in the elderly [5,6]. Approximately 75% of CAP patients require hospitalization, up to 10% of them need to be admitted to the ICU, and the in-hospital mortality from 4 to 20.9% in these patients, which poses a huge burden on families and society [7,8]. Therefore, early identification of elderly CAP patients with high in-hospital mortality is crucial to timely and effective intervention for prognosis improvement.
Several pneumonia severity scores include the Sequential Organ Failure Assessment (SOFA), Pneumonia Severity Index (PSI), CURB-65, and systolic blood pressure, oxygenation, age and respiratory rate (SOAR) have been developed and used to predict outcomes in patients with CAP [9][10][11]. These scores are useful in the management of patient risk stratification, but there is still a lack of accurate assessment with regard to patient mortality [12]. Moreover, a common limitation of the above score systems is that a lot of variables cannot be obtained within the first 24 h after admission [13]. In addition to the scoring methods, in recent years, many blood biomarkers have been shown to play crucial roles in the early diagnosis and prognosis of pneumonia, including CAP [14,15]. Serum albumin (ALB) level was reported to be associated with in-hospital mortality in patients with CAP [12]. The value of the red cell distribution width (RDW) has also been found in predicting the prognosis in critically ill patients [15]. A study demonstrated that the ALB-RDW score is the independent factor of 90-day mortality in patients with severe CAP [16]. Another study indicated neutrophil to lymphocyte (NLR) ratio was a promising candidate predictor of unfavorable outcomes in CAP patients [17]. However, there is a lack of research establishing prediction models to predict in-hospital mortality in CAP patients based on these biomarkers. Therefore, it is necessary to incorporate these biomarkers and develop new prediction models to achieve early assessment of inhospital mortality risk in elderly CAP patients and guide clinical decision-making.
Herein, this study aimed to conduct a prediction model to predict the in-hospital mortality in elderly CAP patients and to compare the predictive value of the prediction model with SOFA, PSI, SOAR, and CURB-65 scoring system. We developed a simpler prediction model that may be beneficial to the decrease of in-hospital mortality in elder patients with CAP.

Study design and population
This study was a retrospective cohort study, and all data were obtained from the Medical Information Mart for Intensive Care III (MIMIC-III) 2001-2012. The MIMIC-III is a large, single-center, freely available database, which contained the comprehensive and high-quality medical records of 50,000 patients admitted to ICU at the Beth Israel Deaconess Medical Center between 2001 and 2012 [18]. Study inclusion criteria were (1) aged ≥ 65 years old; (2) population diagnosed with CAP at the time of admission to ICU. The exclusion criteria of this study were as follows: (1) the number of predictive factors that were not recorded during the first 24 h of the ICU stay exceeds 30% of the total number of predictive factors. Since the clinical data in this study were collected from a publicly available database, there were no local or state ethical issues.

Variable definitions and outcome
CAP was defined as evidence of a pulmonary infiltrate on the chest radiograph and symptoms of lower respiratory infection, including cough, dyspnea, fever, and/or pleuritic chest pain, which were not acquired in a hospital or a nursing home.
PNI referred to 10 × ALB (g/dL) + 0.005 × lymphocytes count. SOFA score uses SBP ≤ 100 mmHg, respiratory rate ≥ 22/min, and altered cognitive state to identify high risk patients. The SOAR score identifies severe CAP using the following criteria (definitions of variables for data extraction are also listed below): SBP < 90 mmHg, PaO2/FiO2 ratio < 250, age ≥ 65, and respiratory rate ≥ 30 breaths/min. The CURB-65 index identifies high risk patients using the following criteria (definitions of variables for data extraction are also listed below): confusion, BUN ≥ 20 mg/dl, respiratory rate ≥ 30 breaths/min, SBP < 90 mmHg or DBP ≤ 60 mmHg, and age ≥ 65.
The study outcome was the 30-day in-hospital mortality rate. When the patient died in the hospital, the followup period ended. The median follow-up was 8 days.

Statistics analysis
The t-test was used to evaluate the normally distributed data which was presented as mean ± standard deviation (Mean ± SD). Mann Whitney U test was used to evaluate the non-normally distributed variables and data were presented as median and quartile M (Q 1 , Q 3 ). Fisher's exact test was used to analyze the enumeration data and data were described as the number of cases and constituent ratio N (%). Multiple imputation was performed by "mice" R package for missing values. Sensitivity analysis was performed by comparing the data before and after imputation.
To establish the robustness of predictor variables, the sample dataset was randomly partitioned into the training set group and the testing set group (ratio: 6.5:3.5). P < 0.05 was considered statistically significant. To determine the predictive factors, variables achieving a significance level were selected for multivariable Logistic analysis and other variables that are significant in other studies were included. Then stepwise regression was performed to construct the final prediction model. The performance of the prediction model was evaluated by the Hosmer-Lemeshow (H-L) goodness-of-fit test, using positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), area under the curve (AUC), and 95% confidence interval (CI). To compare the performance of our prediction model with the SOFA, PSI, SOAR, and CURB-65, the DeLong test was applied. In addition, the decision curve analysis (DCA) was used to assess the net benefit of the prediction model and SOFA, PSI, SOAR, CURB-65. The total population was divided into subgroups to verify our prediction model.
Multiple imputation, DCA curve, and Logistic regression were performed using R Software (version 4.0.3; The R Project for Statistical Computing, TX, USA). ROC curves and prediction results were completed using Python 3.8. All remaining analyses were performed using SAS software (version 9.4; SAS Institute Inc., Cary, NC, USA).

Characteristics of the included patients
In total, 619 patients with CAP aged over 65 years were included in this study, with 217 patients in the testing set and 402 patients in the training set. The median follow-up was 8 days. The flow chart of the participants' selection is shown in Fig. 1. The characteristics of the included patients between the testing set and the training set groups are described in Table 1. Among 402 patients in the training set, 90 (24.63%) elderly CAP patients suffered from 30-day in-hospital mortality; 41.41% (41 of 99) of the patients were male, while 58.59% (58 of 99) were women. The baseline and clinical characteristics between in-hospital mortality and non-in-hospital mortality are present in Table 2.

Identifications of the predictive factors for in-hospital mortality of elderly CAP patients
Factors achieving a significance level in the comparison of the in-hospital mortality and non-in-hospital mortality groups were septic shock (P = 0.003), respiratory failure (P = 0.009), respiratory rate (P = 0.027), SOFA score (P < 0.001), SAPSII score (P < 0.   Table 3.

Prediction model construction and evaluations
Based on the predictive factors, the prediction model for predicting in-hospital mortality of elderly CAP patients was constructed. Table 3 Table 4. The DCA results indicated that the prediction model had a good ability for predicting in-hospital mortality of elderly CAP patients. DCA result of the prediction model is shown in Fig. 2.

Comparison of the predictive performances between the prediction model and SOFA, SOAR, PSI, and CURB-65 scores
We calculated the AUC value of the prediction model and four other scoring systems as shown in Fig. 3. The      Table 4.  Table 5.

Discussion
CAP is a global infectious disease that causes high morbidity and mortality. Accurate and timely identification of patients at high risk of mortality is one of the most important works of physicians. In this study, patients with CAP 30-day mortality rate was 24.63%, slightly higher than the previous related literature reports, this may be because of the large cases included in the ICU, and the age, basic diseases, or a minority of the critically ill patient into the hospital later on. We identified HPR, RDW/BUN ratio, age, respiratory rate, INR, GCS, ventilation use, and vasopressor use were independent predictive factors that were related to the hospital mortality of elderly CAP patients. Based on the predictive factors, our prediction model showed a better predictive performance with an AUC being 0.751. Our model also showed  good goodness-of-fit. The prediction model also demonstrated better predictive performances than the SOFA, PSI, SOAR, and CURB-65 scoring systems. The widespread application of scoring systems in clinical practice has brought great benefits to the management of CAP. However, these assessment scales do have limitations. PSI contains 20 variables and is very complex for prediction, these variables are usually not available at the initial visit, which may affect its promotion and implementation in daily practice. The CURB-65 score may not perform a suitable outcome in patients over the age of 70 due to its low sensitivity [19]. The SOFA score is unduly dependent on clinical therapeutic interventions which fail to be mastered easily [13]. SOAR might be more suitable for assessing disease severity, particularly in the elderly [19]. However, SOAR for in-hospital mortality needs further confirmation. Furthermore, our study showed that these scoring systems have only moderate AUC. Zhang et al. conducted a new prediction model for assessing the clinical outcomes of ICU patients with CAP [20]. However, the AUC of this model was 0.661. Another study with age, congestive heart failure, dementia, respiratory rate and BUN level being predictive factors to develop a prediction model to predict 1-year mortality after hospitalization for CAP. Nevertheless, the C-index was only. 0.76. In addition, the model only focuses on out-of-hospital mortality and only focuses on single biomarkers. The variables considered in our prediction model are more complete. Based on the predictive factors including HPR, RDW/BUN ratio, age, respiratory rate, INR, GCS, ventilation use, and vasopressor use, our prediction model was conducted. our prediction model demonstrated a higher AUC than the scoring systems, with an AUC being 0.751. Moreover, our prediction model was constructed based on the predictive factor that has the benefit of being easily calculated and not dependent on operator capacity to correctly gauge the level of confusion in a patient.
Our study demonstrated that RDW/BUN ratio and HPR were associated with in-hospital mortality of elderly CAP patients. The predictive value of the RDW/BUN Table 4 Performances of the prediction model and scoring systems PPV, positive predictive value; NPV, negative predictive value; AUC, area under the curve; ACC, accuracy; CI: confidence interval; SOFA, Sequential Organ Failure Assessment; SOAR, systolic blood pressure, oxygenation, age and respiratory rate; PSI, Pneumonia Severity Index; * is statistically significant compared with the prediction model The RDW can be obtained immediately from blood routine reports, which has been found to be associated with mortality in patients with CAP [21,22]. Ge et al. also confirmed that elevated RDW and WBC increased mortality in adult CAP patients [23]. Inflammation and oxidative stress caused by infection were thought to be the mechanisms of RDW and infectious diseases. Erythropoietin regulates myelogenesis, red cell maturation, and survival and was previously considered to be one of the major determinants of RDW [24]. Abnormal production of erythropoietin or the body's low response to erythropoietin will lead to a gradual increase in RDW value [25]. CAP is a typical infectious disease, during which Inflammation stimulates the release of inflammatory factors, damages the activity of erythropoietin, prevents the maturation of RBCs, leads to the production of ineffective RBCs, increases the heterogeneity of RBC size, and RDW value [26]. Increased RDW values have been reported to be correlated with inflammatory markers, indicating that C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) are high when RDW is high [27]. BUN is produced by the metabolism of protein and amino acids in the body, then hydrolyzed by the liver and excreted by the kidney with urine [28]. Due to this complex interplay of modulatory factors, BUN is generally used as a surrogate marker of systemic illness rather than a specific marker of renal dysfunction [29]. BUN ≥ 7 mmol was one of the CURB-65 scoring criteria for CAP [30]. In a retrospective study by Kang et al. in China that evaluated 4880 patients aged ≥ 65 with CAP, BUN was a prognostic factor for in-hospital mortality [31]. Uematsu et al. reported that elevated BUN had a significantly higher risk of 30-day mortality in CAP patients [32]. RDW/BUN ratio may be a simple and potentially useful prognostic factor of inhospital mortality in elderly CAP patients. The HPR was calculated based on the hemoglobin and platelet counts. Patients with CAP often exhibit a declining hemoglobin concentration [33]. A study of patients with CAP found  that hemoglobin levels < 10 g/dL were independently associated with 90-day mortality [34]. Abnormal platelet count has previously been related to different complications in patients with CAP admitted to ICU [35]. Rising platelet count throughout hospitalization has been found to be a powerful predictor of better survival, while declining platelet count predicts poor outcomes [36]. Tang et al. found that preoperative HPR can be taken into account as a factor predictive of oncological outcomes for stage 1 and grade 3 bladder cancer, particularly disease progression and mortality outcomes [37]. The HPR can be used as a veritable blood biomarker to predict the in-hospital mortality of elder patients with CAP. Age, respiratory rate, ventilation use, vesopressor use, GCS could also predict the in-hospital mortality of elderly CAP patients in this study. A study evaluating the prognostic factors in hospitalized CAP identified age, respiratory rate, and mechanical ventilation as prognostic factors of in-hospitalized CAP patients [30]. Braunet al. found that variables associated with an increased risk of 90-day mortality included age ≥ 51 years [38]. A study by Baek et al. found that mechanical ventilation was associated with in-hospital mortality of pneumonia [39]. GCS is the most widely accepted tool for evaluating consciousness [40]. Wang et al. identified GCS as an independent predictor that was closely related to the hospital mortality of severe CAP [13].
Our predictive factors are routinely and rapidly measured in patients in a hospital setting; our prediction model therefore may be a useful early tool in predicting elderly CAP patients with a high risk of in-hospital mortality, who require rapid and timely decision making. There are several limitations to this study. First, considering the retrospective design, and the limitation of the sampling analysis, selection and sampling bias cannot be excluded. Second, this study only included hospitalized patients; therefore, it is difficult to generalize these findings to all CAP patients. Third, as no external validation was conducted in this study, the applicability of the prediction model in clinical practice requires further study. Therefore, the results should be interpreted cautiously when applied in other clinical settings. Further multicenter studies with populations of different geographic areas, a larger number of subjects, and above all, a prospective design are needed to corroborate the additive value of these markers to clinical prediction models to provide a safer and more effective assessment tool for clinicians.

Conclusion
HPR, age, respiratory rate, INR, ventilation use, vasopressor use, RDW/BUN ratio, and GCS can be used as the factors to predict the in-hospital mortality of elderly CAP patients. The prediction model based on these predictive factors can help clinicians to make clinical decisions timely and early, and decrease the inhospital mortality of elder patients with CAP admitted to the ICU.